Parallel testing for faster local testing [Deprecated] by shaypal5 · Pull Request #349 · python-cachier/cachier

shaypal5 · 2026-02-28T21:05:23Z

This pull request introduces parallel test execution support to the testing infrastructure, improving test speed and reliability. It adds new pytest fixtures and configuration to enable safe parallelization, including worker-specific isolation for SQL and cache-based tests. The test runner script (test-local.sh) is enhanced with options for parallel execution, and the test dependencies are updated accordingly.

Key changes include:

Parallel Test Execution Support:

Added pytest-xdist and related plugins to tests/requirements.txt to enable parallel test execution and automatic retries for flaky tests.
Implemented new command-line options (-p/--parallel, -w/--workers) in scripts/test-local.sh to run tests in parallel and specify the number of workers. The script now installs pytest-xdist if needed and handles serial vs. parallel test execution, including special handling for pickle and memory tests. [1] [2] [3] [4] [5] [6] [7] [8] [9]

Test Isolation and Fixtures:

Added tests/conftest.py with fixtures for:
- Worker-specific PostgreSQL schema isolation for SQL tests, allowing safe parallelization without table conflicts.
- Isolated cache directories for pickle and maxage tests, preventing cross-worker interference.
- Cleanup of worker-specific schemas after tests complete.
- Custom pytest options for parallel testing.

Configuration Updates:

Updated pyproject.toml:
- Added tool.black configuration for consistent formatting.
- Extended ruff ignores for additional security warnings.
- Added new pytest markers for flaky and serial-local tests.
- Documented and enabled parallel coverage collection.
- Enhanced pytest options and documentation for parallel execution. [1] [2] [3] [4] [5]

These changes collectively make the test suite faster, more robust, and ready for parallel execution, especially important for large or slow-running test suites.

shaypal5 · 2026-02-28T21:09:05Z

@copilot fix precommit-ci errors:

pre-commit.ci
home
docs
pre-commit.com
login
python-cachier / cachier · #349 
queued at: 2026-02-28 21:06:37.638526

total time to completion: 41.8s





fix end of files.........................................................Passed
trim trailing whitespace.................................................Passed
check json...........................................(no files to check)Skipped
check yaml...............................................................Passed
check toml...............................................................Passed
check docstring is first.................................................Passed
check that executables have shebangs.....................................Passed
check for case conflicts.................................................Passed
check for added large files..............................................Passed
detect private key.......................................................Passed
typos....................................................................Passed
docformatter.............................................................Passed
mdformat.................................................................Passed
prettier.................................................................Passed
Black by Ruff............................................................Passed
Ruff check...............................................................Failed
- hook id: ruff-check
- exit code: 1

warning: `incorrect-blank-line-before-class` (D203) and `no-blank-line-before-class` (D211) are incompatible. Ignoring `incorrect-blank-line-before-class`.
warning: `multi-line-summary-first-line` (D212) and `multi-line-summary-second-line` (D213) are incompatible. Ignoring `multi-line-summary-second-line`.
SIM108 Use ternary operator `cache_dir = tmp_path / "cachier_cache" if worker_id == "master" else tmp_path / f"cachier_cache_{worker_id}"` instead of `if`-`else`-block
   --> tests/conftest.py:153:9
    |
151 |       if "pickle" in request.node.keywords:
152 |           # Create a unique cache directory for this test
153 | /         if worker_id == "master":
154 | |             # Not running in parallel mode
155 | |             cache_dir = tmp_path / "cachier_cache"
156 | |         else:
157 | |             # Running with pytest-xdist - use worker-specific directory
158 | |             cache_dir = tmp_path / f"cachier_cache_{worker_id}"
    | |_______________________________________________________________^
159 |
160 |           cache_dir.mkdir(exist_ok=True, parents=True)
    |
help: Replace `if`-`else`-block with `cache_dir = tmp_path / "cachier_cache" if worker_id == "master" else tmp_path / f"cachier_cache_{worker_id}"`

Found 1 error.

mypy.....................................................................Passed
pyproject-fmt............................................................Passed
Validate pyproject.toml..................................................Passed
©[ pre-commit.ci](https://results.pre-commit.ci/)  ·  contact  ·  security  ·  privacy policy  ·  cookie notice  ·  issues

Copilot · 2026-02-28T21:09:12Z

@shaypal5 I've opened a new pull request, #350, to work on those changes. Once the pull request is ready, I'll request review from you.

Copilot

Pull request overview

This PR aims to speed up local development workflows by enabling parallel pytest execution (via pytest-xdist), adding retry support for flaky tests, and documenting the test suite / parallel execution behavior.

Changes:

Add pytest-xdist + pytest-rerunfailures to the base test requirements.
Introduce new pytest fixtures/configuration intended to improve isolation when running tests in parallel (SQL schema isolation, per-worker cache dirs).
Extend scripts/test-local.sh and add test-suite documentation under tests/README.md.

Reviewed changes

Copilot reviewed 5 out of 5 changed files in this pull request and generated 13 comments.

Show a summary per file

File	Description
`tests/requirements.txt`	Adds dependencies for parallel runs / flaky retries (but currently drops an async dependency needed by the suite).
`tests/conftest.py`	Adds fixtures/options to support parallel-safe execution across backends (SQL schema + cache-dir isolation).
`tests/README.md`	Adds comprehensive test-suite and parallel-execution documentation.
`scripts/test-local.sh`	Adds `-p/-w` support and attempts to split out serial local tests while running the rest in parallel.
`pyproject.toml`	Updates pytest/coverage configuration and adds a Black config section.

Comments suppressed due to low confidence (1)

tests/conftest.py:240

The custom --parallel / --parallel-workers options are added here but aren’t used anywhere (no hook that converts them into xdist’s -n behavior). As a result, running pytest --parallel will not actually run tests in parallel. Either implement the wiring (e.g., translate these options into -n) or drop these custom options to avoid a misleading CLI.

        default="auto",
        help="Number of parallel workers (default: auto)",
    )

scripts/test-local.sh

tests/README.md

pyproject.toml

tests/conftest.py

scripts/test-local.sh

tests/README.md

Copilot · 2026-02-28T21:18:37Z

@shaypal5 I've opened a new pull request, #351, to work on those changes. Once the pull request is ready, I'll request review from you.

Copilot · 2026-02-28T21:21:07Z

@shaypal5 I've opened a new pull request, #352, to work on those changes. Once the pull request is ready, I'll request review from you.

Copilot · 2026-02-28T21:22:20Z

@shaypal5 I've opened a new pull request, #353, to work on those changes. Once the pull request is ready, I'll request review from you.

for more information, see https://pre-commit.ci

* Initial plan * Fix SIM108: use ternary operator for cache_dir assignment in conftest.py Co-authored-by: shaypal5 <917954+shaypal5@users.noreply.github.com> --------- Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com> Co-authored-by: shaypal5 <917954+shaypal5@users.noreply.github.com>

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

…flow (#351) Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com> Co-authored-by: shaypal5 <917954+shaypal5@users.noreply.github.com>

* Fix SQL test module path in conftest.py fixture Co-authored-by: shaypal5 <917954+shaypal5@users.noreply.github.com> * Initial plan --------- Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com> Co-authored-by: shaypal5 <917954+shaypal5@users.noreply.github.com>

…354) * Initial plan * Extend isolated_cache_directory fixture to cover maxage-marked tests Co-authored-by: shaypal5 <917954+shaypal5@users.noreply.github.com> --------- Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com> Co-authored-by: shaypal5 <917954+shaypal5@users.noreply.github.com>

* Initial plan * Add pytest-asyncio back to tests/requirements.txt Co-authored-by: shaypal5 <917954+shaypal5@users.noreply.github.com> --------- Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com> Co-authored-by: shaypal5 <917954+shaypal5@users.noreply.github.com>

) * Initial plan * Fix regression: only apply cache isolation in parallel (xdist) mode Co-authored-by: shaypal5 <917954+shaypal5@users.noreply.github.com> * Apply ruff format to conftest.py to fix pre-commit.ci formatting error Co-authored-by: shaypal5 <917954+shaypal5@users.noreply.github.com> --------- Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com> Co-authored-by: shaypal5 <917954+shaypal5@users.noreply.github.com>

shaypal5 · 2026-03-04T10:52:14Z

@Borda Hey! Please take a look and review. :)

tests/conftest.py

Borda

LGTM, though the testing time seems almost the same, and for CI, the most time-consuming steps is start DB in Docker for 5 minutes

~~UPDATE: why don't you run parallel tests also in CI?~~

Borda · 2026-03-06T08:55:43Z

@copilot why don't you run parallel tests also in CI?

Copilot · 2026-03-06T08:55:52Z

@Borda I've opened a new pull request, #359, to work on those changes. Once the pull request is ready, I'll request review from you.

Borda · 2026-03-06T08:56:41Z

@shaypal5 seems pre-commit bot is disabled so not linting fixes are pusched to the PR

* Initial plan * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Add parallel test execution (-n auto) to CI workflow Co-authored-by: Borda <6035284+Borda@users.noreply.github.com> --------- Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Borda <6035284+Borda@users.noreply.github.com>

shaypal5 · 2026-03-06T11:40:20Z

Hey @Borda , please let's finish parallel test execution for local running first, then open a separate PR for its use in the CI.

Please roll back this branch to commit 4a4f2ca, which passed and for which I asked you for a review, and then review it so we can merge it.

Then, you can cherry pick your newer commits to a new branch out of this branch or main, and open a PR when this actually works in the CI. :)

shaypal5 · 2026-03-06T11:58:58Z

Nevermind. Please review PR #360 .

shaypal5 self-assigned this Feb 28, 2026

shaypal5 added enhancement complex issue testing and development CI labels Feb 28, 2026

shaypal5 marked this pull request as draft February 28, 2026 21:05

shaypal5 requested a review from Copilot February 28, 2026 21:05

Copilot started reviewing on behalf of shaypal5 February 28, 2026 21:06 View session

Copilot AI mentioned this pull request Feb 28, 2026

Fix ruff SIM108 lint error in conftest.py #350

Merged

Copilot AI reviewed Feb 28, 2026

View reviewed changes

Copilot AI mentioned this pull request Feb 28, 2026

Fix tests/README.md CI section to match actual GitHub Actions workflow #351

Merged

Copilot AI mentioned this pull request Feb 28, 2026

Fix SQL test module import path in parallel test fixture #352

Merged

Copilot AI mentioned this pull request Feb 28, 2026

[WIP] Update parallel testing implementation based on feedback #353

Closed

3 tasks

shaypal5 had a problem deploying to test February 28, 2026 21:26 — with GitHub Actions Failure

shaypal5 and others added 18 commits March 2, 2026 00:05

small fix

00da69a

[pre-commit.ci] auto fixes from pre-commit.com hooks

ac33217

for more information, see https://pre-commit.ci

Update tests/README.md

1a1c5b9

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

Update scripts/test-local.sh

2d62c99

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

documet flaky test marker in pyproject.toml

123a794

Update pyproject.toml

5f33ab1

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

Update tests/conftest.py

a2032e5

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

Update scripts/test-local.sh

c4548e0

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

Update scripts/test-local.sh

52114a6

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

Update scripts/test-local.sh

5a9cd4d

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

Update scripts/test-local.sh

3cd6183

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

Update scripts/test-local.sh

00f22d6

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

Fix CI/CD Integration section in tests/README.md to match actual work…

c30e2ca

…flow (#351) Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com> Co-authored-by: shaypal5 <917954+shaypal5@users.noreply.github.com>

Borda reviewed Mar 6, 2026

View reviewed changes

tests/conftest.py Outdated Show resolved Hide resolved

Borda reviewed Mar 6, 2026

View reviewed changes

tests/conftest.py Outdated Show resolved Hide resolved

Apply suggestions from code review

fd63d61

Borda approved these changes Mar 6, 2026

View reviewed changes

Copilot AI mentioned this pull request Mar 6, 2026

Enable parallel test execution in CI with -n auto #359

Merged

Conversation

shaypal5 commented Feb 28, 2026 • edited by Borda Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

shaypal5 commented Feb 28, 2026

Uh oh!

Copilot AI commented Feb 28, 2026

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Copilot AI commented Feb 28, 2026

Uh oh!

Copilot AI commented Feb 28, 2026

Uh oh!

Copilot AI commented Feb 28, 2026

Uh oh!

shaypal5 commented Mar 4, 2026

Uh oh!

Uh oh!

Uh oh!

Borda left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Borda commented Mar 6, 2026

Uh oh!

Copilot AI commented Mar 6, 2026

Uh oh!

Borda commented Mar 6, 2026

Uh oh!

shaypal5 commented Mar 6, 2026

Uh oh!

shaypal5 commented Mar 6, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

shaypal5 commented Feb 28, 2026 •

edited by Borda

Loading

Borda left a comment •

edited

Loading